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PREFACE 


This  report  documents  the  results  of  a  study  done  for  AFGWC/WSE  as  USAFETAC  Project 
900919.  The  analyst  was  Capt  Mary  L.  Hart,  USAFETAC/DNE.  The  purpose  was  to  determine 
whether  or  not  reliable  global  Effective  Sunspot  Numbers  (SSNis)  for  AFGWC's  Ionospheric 
Conductivity  and  Electron  Density  (ICED)  model  could  be  calculated  based  on  the  present 
limited  number  (11)  of  ionosonde  sites.  AFGWC  needed  the  results  of  this  study  to  decide 
whether  or  not  their  current  ionosonde  data  network  could  be  used  to  generate  SSNis,  or  if 
additional  Digital  Ionospheric  Sounding  System  (D1SS)  sites  would  have  to  be  added  to  the 
network.  The  study  found  that  increasing  the  number  of  ionosonde  stations  would  hto'e  ?.  limited 
effect  on  the  output  of  today’s  highly  climatological  ICED  model.  It  was  concluded  that  it  was 
feasible  to  run  today’s  ICED  model  using  the  present  11 -station  near-real  time  ionosonde 
network,  subject  to  certain  limitations  described  herein.  Because  ICED  is  not  the  model  Air 
Weather  Service  plans  to  use  for  future  ionospheric  support,  it  is  not  valid  to  use  the  conclusions 
of  this  study  for  determining  the  density  of  future  ionospheric  networks  like  DISS. 
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EFFECTIVE  SUNSPOT  NUMBER  (SSNI)  COMPARISON  STUDY 


1.  INTRODUCTION 

1.1  Purpose.  The  purpose  of  this  study  was  to  determine  whether  or  not  reliable  global 
Effective  Sunspot  Numbers  (SSNis)  for  AFGWC’s  Ionospheric  Conductivity  and  Electron 
Density  (ICED)  model  can  be  calculated  based  on  a  limited  number  of  ionsonde  sites.  Although 
AFGWC/WSE  needs  near-real  time  SSNis  for  ICED,  it  receives  such  data  from  only  11 
ionosonde  sites.  AFGWC  needed  the  results  of  this  study  to  decide  whether  or  not  their  current 
ionosonde  data  network  could  be  used  to  generate  SSNis. 

1.2  The  Quality  Control  Program.  The  ICED  Model  uses  an  effective  sunspot  number 
(SSNi)  as  an  input.  Mr  Herb  K.oeiii  (National  Geophysical  Data  Center-NGDC)  developed  a 
method  for  calculating  SSNis  from  ionosonde  data  using  the  critical  frequency  of  the  F2  layer 
(foF2).  He  provided  AWS  with  two  FORTRAN  programs  that  partially  automate  the  calculation 
process.  The  first  program  (QC)  reads  a  file  of  hourly  foF2  data  for  a  single  ionosonde  site  for  a 
specified  month.  The  program  uses  the  first  day  of  the  month  as  a  reference  day  and  does  a 
day-by-day  comparison  of  foF2’s  for  the  month.  Every  hourly  foF2  that  differs  from  the 
reference  value  by  a  certain  percentage  (25  percent  during  the  day.  50  percent  at  night)  is  flagged 
for  manual  check  by  an  analyst.  QC  also  checks  the  daylight  hours  (06-18  local)  for  significant 
variations  in  foF2  that  signal  storm  activity.  When  ionospheric  storms  occur  during  daylight 
hours,  heating  and  transport  effects  in  the  upper  atmosphere  cause  electron  loss  in  the  F2  layer  of 
the  ionosphere  that  will  tend  to  reduce  observed  foF2s.  When  more  than  half  the  daylight  hours 
differ  front  the  reference  hours  by  more  than  25  percent,  QC  flags  that  day  as  a  storm  day. 

1.3  The  SSNE  Program.  After  the  analyst  checks  flagged  foF2s  manually,  the  second 
program  (SSNE)  calculates  SSNis  from  the  "clean"  foF2  data.  SSNis  are  calculated  from 
ionosonde  data,  using  multiple  ionosonde  site  data  as  input.  The  number  of  sites  used  is 
determined  by  the  size  of  the  geographical  area  the  SSNis  are  meant  to  represent.  When  global 
SSNis  are  calculated,  between  35  and  40  sites  are  commonly  used.  When  calculating  SSNis,  the 
analyst  also  specifies  the  period  of  record  (POR--in  this  case,  usually  a  month),  the  geographical 
area,  and  the  associated  weighting  factors  (two  temporal  and  four  latitudinal  factors).  An  foF2 
weighting  factor  for  each  ionosonde  site  is  also  needed.  SSNE  outputs  hourly  SSNis  for  each 
day  of  the  POR.  The  SSNis  are  effective  sunspot  numbers  developed  specifically  for  use  as 
input  to  ICED. 
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2.  DATA. 


2.1  Hourly  foF2  Data.  Hourly  foF2  data  from  the  USAFETAC  Space  Environmental  Support 
System  (SESS)  Climatic  Database  (SCDB)  for  36  ionosonde  sites  specified  by  AFGWC/WSE 
were  used  in  this  study.  Period  of  record  (POR)  was  from  1  February  to  31  July  1990.  Table  I 
lists  the  36  stations  used  to  provide  the  control  group  of  SSNis  for  this  study.  The  1 1  stations 
used  to  calculate  the  test  SSNis,  a  subset  of  this  group,  are  identified  in  the  table  by  underlining. 
Some  locations  in  the  1 1 -station  subset  (such  as  Resolute  Bay,  Akita,  Wakkanai,  and  Okinawa) 
are  not  full-time  reporting  stations.  Others  in  the  subset  (such  as  Cape  Canaveral  and  Bermuda) 
have  a  fairly  high  number  of  questionable  observations.  The  effects  that  foF2  data  quality  had  on 
the  results  will  be  discussed  in  Section  4,  "Limitations." 

2.2  Equivalent  Auroral  Index.  Another  ICED  input,  the  equivalent  auroral  index  (Qe),  also 
came  from  the  USAFETAC  SCDB.  Because  the  Qe  values  needed  by  ICED  were  not  present  in 
the  SCDB  for  every  hour  of  the  POR,  the  analyst  calculated  a  mean  daily  Qe  for  the  POR.  This 
was  used  with  the  control  and  test  SSNis  as  model  input. 


TABLE  1.  Global  Set  of  Ionosonde  Sites.  Stations  in  the  1 1-station  subset  used  as  the  test 
group  are  underlined. 


Krenkel 

(80.6°  N, 

58.1°  E) 

Nicosia 

(35.2°  N,  33.3°  E) 

Dixon 

(73.5°  N, 

80.4°  E) 

Bermuda 

(32.5°  N,  64.7°  W) 

Murmansk 

uQaT  N, 

33.0"  E) 

Wakkanai 

(45.5°  N,  141.61°  E) 

Salekhard 

(66.5°  N, 

66.7°  E) 

Boulder 

(40.0°  N,  105.3°  W) 

Uppsala 

(59.8°  N, 

17.6°  E) 

Goose  Bay 

(54.3°  N,  60.3°  W) 

Resolute  Bay 

(74.7°  N, 

94.9°  W) 

Vandenberg 

(34.7°  N,  120.6°  W) 

Sverdlovsk 

(56.7°  N, 

61.1°  E) 

Akita 

(39.8°  N,  140.1°  E) 

Tomsk 

(56.5°  N, 

84.9°  E) 

Ottawa 

(45.4°  N,  75.9°  \Y) 

St  Peter 

(54.2°  N, 

8.4°  E) 

Khabarovsk 

(48.5°  N,  135.1°  E) 

College 

(64.9°  N, 

147.8°  W) 

Irkutsk 

(52.5°  N,  104.0°  E) 

Kiev 

(50.7°  N, 

30.3°  E) 

Argentia 

(47.31°  N,  54.0°  W) 

Poitiers 

(46.6°  N, 

0.4°  E) 

Wallops  Is 

(37.9°  N,  75.5°  W) 

Cape  Schmidt 

(68.9°  N, 

179.5°  W) 

Okinawa 

(26.3°  N,  127.8°  E) 

Churchill 

(58.8°  N, 

94.2°  W) 

Cape  Canaveral 

(28.5°  N,  80.6°  W) 

Slough 

(51.5°  N, 

0.6°  W) 

Kahului 

(20.8°  N,  156.5°  W) 

Ashkhabad 

(38.0°  N, 

58.2°  E) 

Taipei 

(25.0°  N,  121.2°  E) 

Magadan 

(60.0°  N, 

151.0°  E) 

Manila 

(14.6°  N,  12 1.0°  E) 

Yakutsk 

(62.0°  N, 

1 29.6°  E) 

Camden 

(34.0°  S,  150.6°  E) 
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3.  METHODOLOGY. 


3.1  Quality  Control.  The  analyst  QC’ed  foF2  data  for  the  36  ionosonde  sites  shown  in  Table 
I,  using  the  process  developed  by  NGDC.  The  SSNE  program  was  used  to  calculate  two  sets  of 
hourly  SSNis  (using  the  36-station  control  and  the  1 !  station  subset  data)  for  each  day  of  the 
POR  (by  month).  SSNi  and  Qe  values  were  used  as  input  to  a  modified  version  of  ICED.  This 
version  of  ICED  calculates  and  outputs  a  series  of  hourly  foF2s  for  a  single  location  over  a 
specified  POR:  it  was  validated  against  test  data  from  AFGWC  during  a  prior  USAFETAC 
Project  (90022601 ).  AFGWC  requested  that  ICED  be  run  for  six  locations  (Vandenberg, 
Sverdlovsk,  Ashkabad,  Cape  Canaveral,  Slough,  and  Magadan),  and  that  the  model  foF2s  for  the 
control  and  test  groups  be  compared  to  foF2s  observed  during  the  POR.  Storm  days  for  each  of 
the  six  locations  were  determined  by  the  QC  program  using  the  criteria  discussed  in  1.2.  A 
census  of  foF2  observations  available  from  each  of  the  six  stations  shown  in  Table  2  shows  the 
number  of  quiet  cases,  storm  cases,  and  all  cases  reported  for  each  hour  during  the  POR.  The 
census  showed  that  there  were  substantial  gaps  in  the  SCDB,  and  that  some  stations  have  almost 
no  storm  cases. 

TABLE  2.  Observed  foF2  Counts  in  SCDB  (POR:  1  Feb-31  Jul  1990). 


Vandenberg  Sverdlovsk  Ashkhabad 

Hr(Z)  Quiet  Storm  All  Quiet  Storm  All  Quiet  Storm  All 


(K) 

139 

33 

172 

126 

18 

144 

135 

4 

139 

01 

138 

33 

171 

127 

19 

146 

154 

3 

157 

02 

138 

34 

172 

125 

21 

151 

148 

4 

152 

03 

134 

31 

165 

126 

25 

151 

140 

3 

143 

04 

139 

31 

170 

123 

22 

145 

131 

1 

132 

05 

134 

32 

166 

123 

20 

143 

127 

3 

130 

06 

132 

31 

163 

118 

16 

134 

126 

2 

128 

07 

136 

31 

167 

130 

27 

157 

137 

2 

139 

08 

132 

31 

163 

126 

148 

138 

4 

142 

09 

136 

33 

169 

122 

22 

144 

139 

4 

143 

10 

137 

33 

170 

119 

23 

142 

137 

2 

139 

11 

135 

33 

168 

125 

20 

145 

119 

3 

122 

12 

132 

34 

166 

112 

20 

132 

133 

3 

136 

13 

132 

35 

167 

122 

21 

143 

130 

3 

133 

14 

135 

32 

167 

114 

22 

136 

114 

2 

116 

15 

140 

33 

173 

125 

23 

148 

116 

4 

120 

16 

140 

32 

172 

127 

23 

150 

97 

3 

MX) 

17 

141 

35 

176 

127 

22 

149 

94 

3 

97 

18 

140 

35 

175 

123 

18 

!4 1 

98 

3 

101 

19 

136 

32 

168 

125 

20 

145 

122 

3 

125 

20 

138 

33 

171 

131 

18 

149 

123 

5 

128 

21 

134 

33 

167 

130 

19 

149 

126 

5 

131 

22 

134 

35 

169 

128 

18 

146 

131 

4 

135 

23 

138 

35 

173 

127 

18 

145 

130 

5 

135 

Total 

3270 

790 

4060* 

29SI 

497 

3478* 

3045 

78 

3123* 

TABLE  2.  Observed  foF2  Counts  in  SCDB  (POR:  1  Feb-31  Jul  1990), 


Cont'd. 


Hr(Z) 

Cape  Canaveral 
Quiet  Storm  All 

Slough 

Quiet  Storm  All 

Magadan 

Quiet  Storm  All 

(X) 

49 

11 

60 

132 

12 

144 

114 

14 

128 

01 

42 

3 

45 

131 

13 

144 

120 

14 

134 

02 

51 

8 

59 

130 

23 

153 

1 14 

15 

129 

03 

76 

7 

83 

129 

13 

142 

1 1 1 

19 

130 

04 

76 

9 

85 

129 

14 

143 

105 

16 

121 

05 

99 

8 

107 

129 

15 

144 

106 

18 

124 

06 

93 

7 

1(K) 

135 

16 

151 

115 

24 

139 

07 

115 

15 

130 

135 

16 

151 

114 

27 

141 

08 

124 

16 

140 

134 

16 

150 

124 

26 

150 

09 

105 

13 

118 

135 

15 

150 

124 

28 

152 

10 

101 

6 

107 

135 

13 

148 

117 

24 

141 

1 1 

48 

6 

54 

137 

15 

152 

112 

25 

137 

12 

70 

1 1 

81 

137 

15 

152 

105 

21 

126 

13 

86 

13 

99 

137 

14 

151 

101 

15 

1 16 

14 

102 

16 

118 

137 

14 

151 

83 

15 

98 

15 

101 

8 

109 

136 

14 

150 

81 

13 

94 

16 

99 

7 

106 

136 

15 

151 

83 

19 

102 

17 

101 

10 

120 

134 

15 

149 

83 

17 

l(X) 

18 

88 

12 

l(K) 

136 

15 

151 

1(H) 

17 

117 

19 

02 

10 

102 

137 

15 

152 

100 

19 

119 

20 

92 

14 

106 

131 

14 

145 

105 

21 

126 

21 

97 

13 

1 10 

127 

15 

142 

105 

26 

131 

22 

104 

14 

118 

133 

15 

148 

112 

25 

137 

23 

82 

13 

95 

131 

15 

146 

108 

23 

131 

Total 

2093 

250 

2343* 

3203 

347 

3550 * 

2543 

4X1 

3024* 

♦Note:  If  there  were  no  missing  observations,  each  station  would 
have  a  total  of  4,344  foF2  values. 

3.2  Calculations.  The  mean,  standard  deviation,  skewness,  and  median  difference  between 
modeled  and  observed  foF2,  and  the  correlation  and  RMS  error  between  modeled  and  observed 
foF2,  were  then  calculated  for  the  six  stations  in  Table  2.  These  statistics  were  calculated  for  the 
following  categories:  the  entire  POR,  the  quiet  days  of  the  POR,  and  the  storm  days  of  the  POR. 
The  normality  of  each  difference  distribution  was  determined  by  a  skewness  test.  The  skewness 
of  each  distribution  was  compared  to  the  standard  deviation  of  a  normal  population  of  the  same 
size,  where  the  standard  deviation  equals  the  square  root  of  6  divided  by  the  number  of 
observations.  If  the  skewness  is  greater  than  three  times  the  standard  deviation,  the  null 
hypothesis  of  normality  is  rejected.  If  the  skewness  is  less  than  three  times  the  standard 
deviation,  the  distribution  may  be  nomial  and  the  assumptions  of  normality  can  be  used. 
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4.  LIMITATIONS. 


4.1  Short  Period  of  Record,  Missing  Observations.  The  main  limitations  in  this  study 
were  the  short  POR  and  the  small  amount  of  data  available  due  to  missing  observations. 
USAFETAC  had  ionosonde  data  covering  a  much  larger  POR,  but  the  amount  of  man  hours 
needed  to  extract  the  data,  QC  it.  and  run  it  through  ICED  made  it  impossible  to  meet  the  quick 
response  required  by  AFGWC/WSE.  The  limitations  imposed  by  the  smaller  POR  were 
acceptable  to  AFGWC/WSE.  Because  the  short  POR  contains  mostly  1990  summer  data,  it 
won't  show  tiie  effect  of  seasonal  or  solar  cycle  variation  in  foF2  on  model  results.  Although 
199(1  was  near  the  peak  of  the  solar  cycle,  the  relative  lack  of  large  flares  and  subsequent  storms 
means  that  the  model  results  are  somewhat  better  than  would  be  the  case  if  the  POR  had 
contained  more  storm  days.  The  small  POR  and  lack  of  storm  days  means  that  most  of  the  six 
study  sites  in  Table  2  have  too  small  a  storm  sample  size  to  provide  any  significant  results, 
especially  when  the  sample  is  subdivided  into  hourly  results.  A  sample  size  of  at  least  30  storm 
days  (720  hourly  observations)  out  of  a  181-day  POR  is  large  enough  to  ensure  that  there  is  a 
low  probability  of  the  results  being  unduly  influenced  by  a  few  extreme  data  values.  The  only 
station  with  enough  storm  days  to  compute  statistics  from  the  hourly  values  was  Vandenberg. 

4.2  foF2  Data  Quality.  Another  limitation  was  the  quality  of  the  foF2  data  reported  by  the  1 1 
sites.  The  problems  with  part-time  stations  and  poor  observation  quality  mentioned  in  2.1 
suggest  that  the  11-station  subset  is  not  really  an  11-station  network.  Since  some  stations, 
notably  Vandenberg,  produce  more  and  better  data,  they  have  a  greater  influence  on  the 
calculated  SSNis. 

4.3  Geographic  Coverage.  Coverage  of  the  1 1 -station  network  was  another  limitation.  The 
eastern  part  of  North  America  has  more  reporting  sites  (Cape  Canaveral,  Wallops  Island,  Goose 
Bay,  Argentia,  and  Bermuda)  than  the  western  part  (Vandenberg.  College,  and  Resolute  Bay). 
The  only  European  site  is  Nicosia,  which  is  closer  to  the  Mideast.  Taipei  and  Manila  tire  the  only 
Asian  sites.  The  1 1 -station  network,  therefore,  does  not  provide  true  global  coverage.  All  these 
limitations  must  be  kept  in  mind  when  interpreting  the  results. 
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5.  RESULTS. 


5.1  Mean,  Median,  and  Standard  Deviation.  Table  3  gives  the  mean,  median,  and 
standard  deviation  of  modeled-observed  foF2  for  all  six  of  the  stations  studied.  The  Vandenberg 
storm  distributions  for  both  the  global  and  subset  case  pass  the  skewness  test  for  normal 
distributions.  The  quiet  days  distribution  is  skewed,  but  does  center  on  the  origin.  The 
distributions  for  all  days  can  be  assumed  to  be  normal.  The  Vandenberg  distributions  for  quiet 
days  and  all  days  show  a  mean  and  median  slightly  greater  than  zero.  A  difference  in  accuracy 
of  only  2  percent  (calculated  by  taking  the  absolute  value  of  the  global  -  subset  mean,  divided  by 
the  larger  of  the  two  means  times  100)  exists  between  the  global  and  subset  groups.  Sverdlovsk 
has  distributions  similar  to  Vandenberg’s,  but  only  the  storm  days  distributions  pass  the 
skewness  test.  The  difference  between  the  subset  group  and  the  global  group  means  is  12 
percent  for  the  worst  case. 

TABLE  3.  (Modeled-Observed)  foF2  (MHz)  Distributions. 


Vandanbarg 

Svardlovak 

Ashkhabad 

QUIET  DAYS 

Global  Mean 

0.02 

0.15 

-0.01 

Subset  Mean 

0.02 

0.14 

-0.02 

Global  St  Dev 

1.20 

1.10 

0.98 

Subset  St  Dev 

1.20 

1.20 

1 . 00 

Global  Median 

0.05 

0.24 

0.01 

Subset  Median 

0.06 

0.25 

-0.03 

STORM  DAYS 

Global  Mean 

0.94 

0.98 

— 

Subset  Mean 

0.92 

1 .10 

— 

Global  St  Dev 

1  .  40 

1.20 

— 

Subset  St  Dev 

1.30 

1 .  40 

— 

Global  Median 

0.89 

0.93 

— 

Subset  Median 

0.87 

1  .  00 

— 

ALL  DAYS 

Global  Mean 

0.20 

0.27 

0.01 

Subset  Mean 

0.20 

0.28 

0.01 

Global  St  Dev 

1.30 

1 . 20 

1 . 00 

Subset  St  Dev 

1.30 

1 . 30 

1.00 

Global  Median 

0.18 

0.34 

0.02 

Subset  Median 

0.20 

0.  37 

-0.01 
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TABLE  3.  (Modeled-Observed)  foF2  (MHz)  Distributions,  Cont’d 


CSE 

•  Canaveral 

Slouoh 

Naqadan 

QOXET  DAYS 

Global  Mean 

-0.41 

-0.31 

-0.01 

Subset  Mean 

-0.41 

-0.37 

-0.04 

Global  St  Dev 

0.99 

0.87 

1.10 

Subset  St  Dev 

0.94 

1 . 00 

1.10 

Global  Median 

-0.36 

-0.32 

-0.07 

Subset  Median 

-0.39 

-0.39 

-0.13 

STORM  DAYS 

Global  Mean 

0.50 

0 . 69 

0.81 

Subset  Mean 

0.42 

1 .00 

0.94 

Global  St  Dev 

1.70 

1.30 

1 .10 

Subset  St  Dev 

1.50 

1 . 60 

1 .20 

Global  Median 

0.50 

0.71 

0.76 

Subset  Median 

0.42 

0.99 

0.85 

ALL  DAYS 

Global  Mean 

-0.31 

-0.21 

0.12 

Subset  Mean 

-0.32 

-0.23 

0.12 

Global  St  Dev 

1.12 

0.97 

1.12 

Subset  St  Dev 

1.05 

1.16 

1.20 

Global  Median 

-0.31 

-0.25 

0.06 

Subset  Median 

-0.33 

-0.30 

0.01 

5.2  Modeled-Observed  foF2  for  Vandenberg  and  Sverdlovsk.  Figures  la  and  lb  show 
distributions  of  Modeled-Observed  foF2  for  Vandenberg  and  Sverdlovsk,  respectively.  Results 
from  the  control  group  (referred  to  from  now  on  as  the  "global  group")  are  at  the  top,  and  results 
from  the  test  group  (now  called  the  "subset  group")  are  at  the  bottom.  The  dark  histograms  show 
results  for  storm  days;  the  lighter  histograms,  quiet  days.  The  storm  distributions  are  displaced 
to  the  right  of  the  origin  (X=(),  Y=0),  indicating  that  the  modeled  foF2s  tend  to  be  larger  than  the 
observed  foF2s. 


Figure  la.  Distributions  of  Modeled-Observed  foF2  {MHz)  for  Vandenberg. 
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5.3  Modeled-Observed  foF2  for  Ashkabad  and  Cape  Canaveral.  Figures  2a  and  2b 
show  foF2  distributions  for  Ashkhabad  and  Cape  Canaveral,  respectively.  See  Table  3  for 
means,  standard  deviations,  and  medians.  Because  Ashkhabad  has  almost  no  storm  days,  those 
results  are  not  included  in  the  table.  The  quiet  days  and  all  days  distributions  are  normal  for  the 
subset  group,  but  not  for  the  global  group.  The  means  for  both  groups  are  slightly  less  than  zero. 
The  worst  case  percent  difference  between  the  global  and  subset  means  seems  large  (50  percent), 
but  it's  due  to  the  small  values.  Cape  Canaveral  has  more  storm  days  than  Ashkhabad,  but  not 
nearly  as  many  as  Vandenberg.  Cape  Canaveral  has  a  history  of  poor  data  quality  (according  to 
NGDC)  and  the  results  should  be  used  with  caution.  The  means  and  medians  for  both  the  global 
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and  subset  groups  are  negative  or  close  to  zero,  indicating  that  the  model  is  under-predicting 
foF2s.  The  reason  for  this  trend  is  not  apparent.  Only  the  subset  distribution  for  the  quiet  days 
can  be  considered  normal. 


PERCENT 


Ashkhabad  Difference  Distribution 
Global 


-5.0  -4.0 


PERCENT 


-3.0  -2.0  -1.0  0  +1.0  +2.0  +3.0  +4.0  +5.0 

-  MHz  - 

Ashkhabad  Difference  Distribution 
Subset 


+3.0  +4.0  +5.0 


STORM 


Figure  2a.  Distributions  of  Modeled-Observed foF2  (MHz.)  for  Ashkabad. 
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Cape  Canaveral  Difference  Distribution 
Global 


-S-W  -4.0  -3.0  -2.0  -1.0  0  +1.0  +2.0  +3.0  +4.0  +5.0 

-MH*- 

Cape  Canaveral  Difference  Distribution 
PERCENT  Subsot 


Figure  2b.  Distributions  of  Modeled-Observed foF2  (MHz)  for  Cape  Canaveral. 

5.4  Modeled-Observed  foF2  for  Slough  and  Magadan.  Figures  3a  and  3b  give  foF2 
distributions  for  Slough  and  Magadan,  respectively;  see  Table  3  for  means,  medians,  and 
standard  deviations.  Storm  days  and  quiet  days  distributions  for  both  the  global  and  subset 
groups  are  normal.  The  storm  distributions  are  displaced  to  the  right  of  the  origin  (also  shown  by 
the  positive  mean  and  median),  indicating  that  the  model  is  again  over- forecasting  foF2.  The 
observation  count  for  stomi  days  is  still  too  small,  with  less  than  half  the  preferred  count  of  at 
least  720  observations.  Distributions  for  all  days  are  skewed  for  both  the  global  and  subset 
groups.  The  model  slightly  under-predicts  foF2  in  both  the  quiet-days  and  all-days  cases.  The 
worst  case  difference  between  global  and  subset  results  for  storm  days,  is  31  percent.  This  is  the 
worst  result  of  all  the  stations.  The  results  for  quiet  days  and  all  days  are  somewhat  better,  with  a 
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worst  case  difference  of  16  percent.  The  lack  of  European  sites  in  the  subset  network  contributes 
to  these  poor  results.  Magadan  has  more  storm  days  than  Slough,  and  just  over  half  the  number 
of  observations  from  Vandenberg.  All  distributions  from  Magadan  are  skewed.  The  largest 
difference  between  global  and  subset  results  occurs  in  the  quiet  days  distributions,  but  the  large 
difference  (75  percent)  is  due  to  the  fact  that  the  means  are  very  close  to  zero.  The  storm  days 
case  has  a  difference  of  14  percent  between  the  global  and  subset  data. 


Figure  3  b.  Distributions  of  Modeled-Observed  foF2  (MHz)  for  Magadan. 


5.5  Summary  Of  Distribution  Results.  The  global  and  subset  distributions  for  each  of 
the  six  sites  are  very  similar,  indicating  that  the  subset  SSNis  generally  gave  reasonable  results. 
While  the  quiet  days  and  all  days  distributions  (in  Table  1)  look  fairly  close  to  normal,  they  are 
really  skewed  in  half  the  cases,  as  the  differences  between  mean  and  median  values  also  show. 
When  using  the  means  and  standard  deviations  of  the  differences  between  modeled  and  observed 
foF2  to  judge  the  accuracy  of  the  global  and  subset  results,  this  fact  must  be  kept  in  mind.  The 
storm  days  distributions  are  not  skewed,  but  they  may  not  be  truly  normal  eitnei.  The  flattened 
shape  of  the  distributions,  with  the  larger  number  of  values  toward  the  extremes,  has  to  be  kept  in 
mind  when  evaluating  these  distributions.  Since  Vandenberg  has  the  largest  number  of  storm 
days  of  the  six  sites,  its  results  should  be  given  the  greatest  consideration.  Next,  we  look  at  the 
Vandenberg  results  in  more  detail,  breaking  down  the  distributions  by  Zulu  hour. 
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5.6.  Detailed  Results  for  Vandenberg.  Figures  4a  (storm  days)  and  4b  (quiet  days)  show 
Vandenberg’s  mean  (x),  standard  deviation,  and  median  (0)  of  the  difference  between  modeled 
and  observed  foF2  for  each  hour  of  the  day.  The  plots  for  the  global  and  subset  data  are  very 
similar  here,  as  w’ell.  For  the  storm  days  case,  both  global  and  subset  plots  show  the  distribution 
centered  above  the  zero-error  line  (with  means  between  0.21  and  1.83  MHz),  indicating  that  the 
model  over-predicts  foF2.  This  is  true  for  all  hours  of  the  day,  but  is  especially  true  for  daylight 
hours  (l^et ween  06  and  18  local,  or  14  to  02Z).  The  daytime  standard  deviations  are  also  larger 
than  those  at  night,  ranging  from  0.70  to  1.71  MHz.  There  are  also  small-scale  fluctuations  that 
take  place  over  a  period  of  several  hours,  particularly  during  nighttime  hours. 


Figure  4a.  Means  (jc),  Medians  (0),  and  Standard  Deviations  of  Distributions  of  Modeled- 
Observed  Storm-Days  foF2  (MHz)  for  Vandenberg  by  Zulu  Hour. 
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Quiet  day  distributions  are  similar  for  global  and  subset  data.  These  distributions  are  centered 
more  closely  on  the  zero-error  line  and  show  no  diurnal  variation.  Here  too,  small-scale 
fluctuations  with  a  period  of  several  hours  can  be  seen.  The  fluctuations  are  almost  symmetric 
about  the  zero  line;  means  range  between  -0.43  and  0.55  MHz.  In  this  case,  standard  deviations 
show  no  diurnal  change,  ranging  between  1.0  and  1.4  MHz. 


Mean  +  SD 
Mean/Mediar 
Mean  -  SD 


Mean  +  SD 
Mean/Median 
Mean  -  SD 


Figure  4b.  Means  (x),  Medians  (0),  and  Standard  Deviations  of  Distributions  of  Modeled- 
Observed  Quiet-Days  foF2  (MHz)  for  Vandenberg  by  Zulu  Hour. 
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5.6.1  The  model’s  overprediction  of  foF2  on  storm  days  can  be  seen  in  Figures  5  and  6,  which 
show  hourly  foF2  distributions  for  Vandenberg.  Figures  5a,  5b,  and  5c  show  observed  foF2  for 
quiet  days,  storm  days,  and  all  days,  respectively.  Figures  6a  amd  6b  show  modeled  foF2 
distributions  for  quiet  days  and  storm  days,  respectively.  The  global,  subset,  and  observed  foF2s 
all  show  diurnal  variation  for  the  quiet  days  case,  but  observed  foF2s  on  storm  days  show  almost 
no  diurnal  variation-daytime  values  are  almost  the  same  as  nighttime  values.  This  is  not  the 
case  in  the  model  results,  where  diurnal  variations  in  foF2  are  a  little  more  pronounced.  The  lack 
of  a  diurnal  curve  in  observed  foF2  should  not  be  a  surprise,  given  the  methodolgy  used  to 
determine  storm  days  with  the  QC  program,  which  screens  the  daylight  hours  for  foF2  values 
outside  the  range  of  expected  values.  The  depletion  of  F-layer  electrons  by  storm  activity  means 
that  daytime  foF2  values  are  lower  than  normal  on  storm  days. 

5.6.2  Global  and  subset  modeled  foF2  distributions  are  almost  identical  to  each  other  for  both 
quiet  and  storm  days.  The  observed  and  modeled  foF2  distributions  also  show  the  classic 
summertime  pattern,  with  less  diurnal  variation  in  foF2  than  in  winter  months.  These 
distributions  are  evidence  that  the  winter  months  were  not  well  represented  in  the  data  sample, 
and  that  the  results  don’t  show  how  accurate  the  subset  SSNis  are  then. 


Mean  +  SD 
Mean/Mediar 
Mean  -  SD 


Figure  Sa.  Hourly  Distributions  of  Observed  foF2  {MHz)  for  Vandenberg,  Quiet  Days . 


16 


Observed  foF2 


00  01  02  03  04  05  06  07  08  09  10  11  12  13  14  15  16  17  18  19  20  21  22  23 

Zulu  Hour 


Modeled  foF2  Modeled  foF2 


Vandenberg  Quiet  Days 
Global 


Subset 


Mean  +  SD 
Mean/Mediar 
Mean  -  SD 


Zulu  Hour 


Mean  +  SD 
Mean/Median 
Mean  -  SD 


Figure  6a.  Hourly  Distributions  of  Modeled  foF2  (MHz)  for  Vandenberg,  Quiet  Days. 
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Figure  6b.  Hourly  Distributions  of  Modeled  foF2  (MHz)  for  Vandenberg,  Storm  Days. 
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5.7  Correlations.  Table  4  shows  correlations  between  observed  and  modeled  foF2  for  all  six 
sites.  Figure  6  shows  hourly  correlations  between  observed  and  modeled  foF2  for  Vandenberg. 


5.7.1  Both  global  and  subset  model  runs  produced  foF2  values  that  correlated  well  with 
observed  foF2s,  as  can  be  seen  in  Table  4.  Correlations  over  the  entire  POR  (for  quiet  and  storm 
days)  are  between  0.84  and  0.92  for  the  global  data,  and  between  0.80  and  0.91  in  the  subset 
case.  Subset  correlations  are  usually  a  little  less  than  for  the  global  data.  In  the  cases  of 
Vandenberg  and  Cape  Canaveral,  the  subset  correlations  are  the  same  or  slightly  higher.  The 
Cape  Canaveral  results  must  be  viewed  with  caution,  however,  since  that  station’s  observations, 
according  to  NGDC,  can  be  unreliable. 


TABLE  4.  Correlations  Between  Modeled  and  Observed  foF2. 


Global  SSNis  Subset  SSNis 


Station 

Quiet 

Storm 

All 

Quiet 

Storm 

All 

Vandenberg 

0.904 

0.636 

0.881 

0.905 

0.648 

0.883 

Sverlovsk 

0.900 

0.707 

0.876 

0.876 

0.660 

0.843 

Ashkhabad 

0.921 

0.918 

0.918 

0.914 

Cape  Canaveral 

0.897 

0.236 

0.862 

0.909 

0.308 

0.881 

Slough 

0.935 

0.754 

0.912 

0.895 

0.654 

0.854 

Magadan 

0.860 

0.642 

0.835 

0.838 

0.610 

0.804 

5.7.2  Correlations  on  storm  days  are  always  lower  than  on  quiet  days,  sometimes  considerably 
lower.  The  relatively  small  sample  sizes  on  storm  days  limit  the  usefulness  of  those  results. 
Figures  7a-c  show  hourly  correlations  for  Vandenberg,  the  only  station  with  a  large  enough 
sample  size  for  hourly  breakdowns.  The  correlations  are  lower  during  the  nighttime  hours  for 
quiet  days  (7a),  storm  days  (7b),  and  all  days  (7c).  In  the  storm  days  case,  the  difference 
between  the  global  and  subset  data  is  also  greater  during  nighttime  hours.  In  general,  subset 
correlations  are  higher  than  global  correlations  during  the  early  hours  of  the  night,  while  global 
correlations  are  higher  than  subset  correlations  in  the  hours  just  before  sunrise.  Daytime 
correlations  are  virtually  the  same  for  both  global  and  subset  data.  Global  and  subset  correlations 
are  also  very  similar  for  all  hours  in  the  quiet  days  (and  therefore  all  days)  cases. 
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Figure  7a.  Hourly  Correlations  Between  Modeled  and  Observed  foF2  (MHz)  for  Vandenberg, 
Quiet  Days. 


Figure  7b.  Hourly  Correlations  Between  Modeled  and  Observed  foF2  (MHz)  for  Vandenberg, 
Storm  Days. 


5.8  RMS  Error.  Table  5  shows  the  six-station  RMS  errors  (for  the  global  and  subset  data), 
subdivided  into  columns  for  quiet  days  (left),  storm  days  (middle),  and  all  days  (right)  cases. 
The  range  of  RMS  errors  for  the  six  sites  is  small:  between  0.99  and  1.9  MHz.  All  the  sites 
except  Vandenberg  and  Cape  Canaveral  show  higher  RMS  errors  in  the  subset  data,  but  the 
difference  in  RMS  between  the  global  and  subset  data  is  always  less  than  1  MHz.  In  the  case  of 
Vandenberg,  the  differences  between  global  and  subset  RMS  is  0.02  MHz  or  less.  The 
differences  between  global  and  subset  data  are  larger  in  the  case  of  Cape  Canaveral,  but  the 
observations  from  this  station  are  less  reliable  than  those  from  Vandenberg.  The  largest 
difference  between  the  global  and  subset  data,  for  Slough,  is  still  less  than  0.5  MHz.  None  of  the 
differences  in  RMS  error  are  large  enough  to  indicate  a  problem  in  the  ICED  model’s  accuracy, 
whichever  type  of  SSNi  is  used  as  input. 

TABLE  5.  RMS  Errors  (MHz)  for  Modeled-Observed  foF2. 


Station 

Global  SSNis 

Quiet  Storm  All 

Subset  SSNis 

Quiet  Storm  All 

Vandenberg 

1.20 

1.64 

1.30 

1.19 

1.62 

1.29 

Sverdlovsk 

1.11 

1.56 

1.19 

1.20 

1.79 

1.30 

Ashkhabad 

0.99 

— 

1.01 

1.00 

— 

1.03 

Cape  Canaveral 

1.07 

1.75 

1.16 

1.02 

1.57 

1.09 

Slough 

0.92 

1.47 

0.99 

1.08 

1.90 

1.19 

Magadan 

1.07 

1.38 

1.13 

1.13 

1.54 

1.21 

5.9  RMS  Errors— Vandenberg.  The  hourly  RMS  errors  for  Vandenberg  are  shown  in  Figure 
8a  (quiet  days),  8b  (storm  days),  and  8c  (all  days).  Quiet  days  and  all  days  plots  show  no  diurnal 
variation  in  RMS  errors,  and  the  errors  remain  in  a  narrow  range  (1.0  to  1.5  MHz).  There  are 
some  small-scale  fluctuations  in  the  RMS,  but  these  don’t  appear  to  be  tied  to  the  diurnal  cycle. 
The  global  and  subset  RMS  errors  are  virtually  identical.  The  storm  days  plot  shows  more  of  a 
diurnal  variation  in  RMS,  with  errors  increasing  near  sunrise  and  remaining  relatively  high 
during  daylight  hours.  Errors  range  from  about  0.8  MHz  at  night  and  2.3  MHz  during  the  day. 
This  is  expected,  since  the  model  is  less  accurate  during  daylight  hours  of  storm  days.  There  are 
also  indications  of  smaller-scale  fluctuations  in  RMS  similar  to  quiet  days  and  all  days  cases. 
The  difference  between  the  global  and  subset  data  is  a  little  more  pronounced  than  in  the  other 
two,  but  is  still  not  significant.  Of  greater  concern  is  the  relatively  high  daytime  RMS  Error.  A 
study  using  a  more  extensive  POR  would  be  needed  to  determine  whether  this  problem  is 
important  enough  to  justify  some  sort  of  adjustment  to  the  way  the  ICED  model  calculates  foF2s 
for  storm  days. 
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6.  CONCLUSIONS. 


6.1  Main  Conclusion.  We  can  conclude  from  this  limited  study  that  using  foF2  data  from  the 
1 1 -station  subset  to  calculate  real  time  SSNis  does  not  significantly  reduce  the  accuracy  of  ICED 
output.  That  conclusion  is  supported  by: 

•similarities  between  global  and  subset  SSNis  for  distributions  in  modeled-observed  foF2. 

•similarities  between  global  and  subset  SSNis  for  hourly  distributions  in  modeled-observed 
foF2  at  Vandenberg. 

•similarities  in  the  correlations  between  observed  and  modeled  foF2. 

•similarities  between  RMS  errors  for  global  and  subset  SSNis. 

The  worst  results  in  all  the  statistics  were  seen  during  storm  days,  but  the  limited  amount  of  data 
makes  it  hard  to  draw  any  definite  conclusions  about  those  results. 

6.2  Geographical  Bias.  The  geographical  bias  toward  North  America  in  the  11 -station 
ionosonde  network  explains  why  the  results  for  Vandenberg  and  Cape  Canaveral  tended  to  be 
better  than  for  the  European  and  Soviet  sites.  Eleven  sites  may  be  enough  to  calculate  reasonable 
global  SSNis  for  use  with  ICED,  but  the  ICED  output  for  regions  like  Europe  could  be  improved 
with  a  network  that  covers  the  globe  more  evenly. 

6.3  Summary  Conclusion.  Increasing  the  number  of  digital  ionosonde  sites  would  have  a 
limited  effect  on  ICED  output  because  of  the  limitations  of  an  empirical  model  like  ICED  and  the 
concept  of  an  effective  sunspot  number  as  a  way  to  initialize  the  model.  The  ICED  model  is  not 
very  responsive  to  changes  in  the  the  SSNi  input.  However,  these  results  are  not  applicable  to  a 
physical  model  like  the  Parameterized  Real-time  Ionospheric  Specification  Model  (PRISM), 
which  initializes  itself  directly  from  data  instead  of  from  a  proxy  like  SSNi.  These  results,  in 
particular,  cannot  be  used  to  draw  conclusions  about  the  necessary  density  of  ionospheric 
sounding  or  sensing  systems.  We  are  confident  of  our  results  within  the  limitations  discussed 
earlier. 
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SPECIALIZED  TERMS  AND  ACRINABs 


ACRINABs 

AFGWC 

AWS 

DISS 

F2  layer 

foF2 

ICED 

MHz 

NGDC 

POR 

PRISM 

RMS  error 

Qe 

QC 

SESS 

SCDB 

SSNE 

SSNi 

USAFETAC 

WSE 

Z 


acronym,  initialisms,  and  abbreviations 
Air  Force  Global  Weather  Central 
Air  Weather  Service 
Digital  Ionospheric  Sounding  System 

layer  in  the  ionosphere  at  which  electron  density  reaches  its  maximum 
critical  frequency  of  the  F2  layer  (highest  electron  density) 

Ionospheric  Conductivity  and  Electron  Density  Model 
megahertz,  a  frequency  measurement 
National  Geophysical  Data  Center 
period  of  record 

Parameterized  Real-time  Ionospheric  Specification  Model 
root  mean  square  error 

equivalent  Q  index,  calculated  by  AFGWC/WSE  to  measure  size  of  auroral 
oval 

FORTRAN  program  that  quality  controls  foF2  data 
Space  Environmental  Support  System 
SESS  Climatic  Database 

FORTAN  program  that  calculates  effective  sunspot  numbers 
Effective  sunspot  number  for  use  with  the  ICED  model 
USAF  Environmental  Technical  Applications  Center 
AFGWC  Space  Environmental  Support 
Zulu  or  Greenwich  Mean  Time 
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